Learning Partial Differential Equations for Computer Vision ∗

Partial differential equations (PDEs) have been successful for solving many problems in computer vision. However, the existing PDEs are all crafted by people with skill, based on some limited and intuitive considerations. As a result, the designed PDEs may not be able to handle complex situations in real applications. Moreover, human intuition may not apply if the vision task is hard to describe, e.g., object detection. These two aspects limit the wider applications of PDEs. In this paper, we propose a framework for learning a system of PDEs from real data to accomplish a specific vision task. As the first study on this problem, we assume that the system consists of two PDEs. One controls the evolution of the output. The other is for an indicator function that helps collect global information. Both PDEs are coupled equations between the output image and the indicator function, up to their second order partial derivatives. The way they are coupled is suggested by the shift and the rotational invariance that the PDEs should hold. The coupling coefficients are learnt from real data via an optimal control technique. Our framework can be extended in multiple ways. The experimental results show that learning-based PDEs could be an effective regressor for handling many different vision tasks. It is particularly powerful for those tasks that are difficult to describe by intuition.